System and method for providing a two-part graphic design and interactive document application

ABSTRACT

A method and system creates an interactive multimedia document that incorporates references to multimedia files that support a particular logical step or argument presented within the document. The method and system utilizes several modules including a builder module and a presenter module. The references are configured to determine, upon selection, a visual format and a relevant portion of the multimedia file to be displayed. Thus, only one copy of the multimedia file needs to be incorporated into the document. The multimedia files include text documents, portable document files, PowerPoint® presentations, audio files, video files, voice narrated graphics, and other types of files. In some embodiments, the interactive multimedia document is a legal brief and the multimedia files include depositions, supporting case law, evidence, and other materials.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.12/050,049, filed Mar. 17, 2008, the disclosure of which is herebyincorporated herein by reference in its entirety for all purposes.

TECHNICAL FIELD

The following disclosure relates to a system and a method for providinga two-part graphic design and interactive media application thatincludes a builder module to create digital interactive multimediadocuments and a presenter module to provide a seamless, self-guidedviewer experience with access to linked media files.

BACKGROUND

Conventional documents incorporate media files into a single flatteneddocument that is typically in PDF format. Conventional methods alsorequire a user to use multiple applications, such as, for example,Adobe™, Photoshop™, Illustrator™, Acrobat™, and Flash™, and multiplecopies of media files referenced by hyperlinks to allow navigation tothe linked documents. However, in order to follow hyperlinks and otherreferences to media files, the viewers must follow the referencecompletely, and the viewer is “relocated” to the linked file so that theprimary focus of the user is drawn away from the document that is thesource of the reference, to the media file that merely supports thedocument. Thus, there is no technology that exists to allow the designand presentation of focused dynamic documents that incorporate mediafiles in various formats within a single application and within a singleinstance of the document.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of one example of a network and network devicesincluding a user access point and a web-based information managementsystem;

FIG. 2 is a diagram of one example of a general computing device thatmay operate in accordance with the claims;

FIG. 3 is a block diagram illustrating the interrelationships between abuilder screen and a presenter screen.

FIG. 4 is a flowchart describing a method of one example of building adynamic interactive multimedia document;

FIG. 5 is a flowchart describing an alternative method of building adynamic interactive multimedia document;

FIGS. 6-14 are screen shots of a viewer's view of an exemplary dynamicinteractive multimedia document.

DETAILED DESCRIPTION

FIG. 1 illustrates an example of a network typical of the World WideWeb. A network 10 may be a virtual private network (VPN), or any othernetwork that allows one or more computers, communication devices,databases, etc., to be communicatively connected to each other. Thenetwork 10 may be connected to a PC 12 or a computer terminal 14 by anymeans able to communicate electronic signals. In one embodiment, thecomponents may be connected to the network 10 via an Ethernet 16 and arouter 20, or a land line 22. The network 10 may also be wirelesslyconnected to a laptop computer 24 and a personal data assistant 26 via awireless communication station 30 and a wireless link 32. Similarly, aserver 34 may be connected to the network 10 using a communication link36. Also, an information management system 40 may be connected to thenetwork 10 using another communication link 42. Where the network 10includes the Internet, data communication may take place over thenetwork 10 via an Internet communication protocol. In operation, theclient PC 12 may view or request data from any other computing deviceconnected to the network 10. Further, the PC 12 may send data to anyother computing device connected to the network 10.

FIG. 2 illustrates a typical computing device 50 that may be connectedto the network 10 of FIG. 1 and participate in a distributed computingenvironment such as the World Wide Web and communicate with aninformation management system 40. FIG. 2 may also be an example of anappropriate computing system on which the claimed apparatus and claimsmay be implemented, however, FIG. 2 is only one example of a suitablecomputing system and is not intended to limit the scope or function ofany claim. The claims are operational with many other general or specialpurpose computing devices such as PCs 12, server computers 34, portablecomputing devices such as a laptop 24, consumer electronics 26,mainframe computers, or distributed computing environments that includeany of the above or similar systems or devices.

With reference to FIG. 2, a system for implementing the steps of theclaimed apparatus may include several general computing devices in theform of a computer 50. The computer 50 may include a processing unit,51, a system memory, 52, and a system bus 54 that couples various systemcomponents including the system memory 52 to the processing unit 51. Thesystem bus 54 may include an Industry Standard Architecture (ISA) bus, aMicro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, a PeripheralComponent Interconnect (PCI) bus or a Mezzanine bus, and the PeripheralComponent Interconnect Express (PCI-E) bus.

Computer 50 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 110 and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes both volatileand nonvolatile, removable and non-removable media implemented in anymethod or technology for storage of information such as computerreadable instructions, data structures, program modules or other data.Computer storage media includes, but is not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can accessed by computer 50. Communication media typicallyembodies computer readable instructions, data structures, programmodules or other data in a modulated data signal such as a carrier waveor other transport mechanism and includes any information deliverymedia. The term “modulated data signal” means a signal that has one ormore of its characteristics set or changed in such a manner as to encodeinformation in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of the any of the aboveshould also be included within the scope of computer readable media. Thesystem memory 52 may include storage media in the form of volatileand/or non-volatile memory such as ROM 56 and RAM 62. A basicinput/output system 60 (BIOS), containing algorithms to transferinformation between components within the computer 50, may be stored inROM 56. Data or program modules that are immediately accessible or arepresently in use by the processing unit 51 may be stored in RAM 62. Datanormally stored in RAM while the computer 50 is in operation may includean operating system 64, application programs 66, program modules 70, andprogram data 72. The program modules 70 may also include, for example,an interactive multimedia document module 71 to create and viewinteractive multimedia documents, as further described below. Theinteractive multimedia document module 71 may also include any number ofsub-modules, for example, a builder module 71 a, and a presenter module71 b that may be used by the interactive multimedia document module 71to create interactive multimedia documents using the computer 50.Additionally, each sub-module may include any number of modulesincluding, for example, an embedding module of the builder module 71 aand a video module of the presenter module 71 b.

The system memory 52 may include storage media in the form of volatileand/or non-volatile memory such as ROM 56 and RAM 62. A basicinput/output system 60 (BIOS), containing algorithms to transferinformation between components within the computer 50, may be stored inROM 56. Data or program modules that are immediately accessible or arepresently in use by the processing unit 51 may be stored in RAM 62. Datanormally stored in RAM while the computer 50 is in operation may includean operating system 64, application programs 66, program modules 70, andprogram data 72.

The computer 50 may also include other storage media such as a hard diskdrive 76 that may read from or write to non-removable, non-volatilemagnetic media, a magnetic disk drive 92 that reads from or writes to aremovable, non-volatile magnetic disk 94, and an optical disk drive 96that reads from or writes to a removable, nonvolatile optical disk 96.Other storage media that may be used includes magnetic tape cassettes,flash memory cards, digital versatile disks, digital video tape, solidstate RAM, and solid state ROM. The hard disk drive 76 may be connectedto the system bus 54 through a non-removable memory interface such asinterface 74. A magnetic disk drive 92 and optical disk drive 96 may beconnected to the system bus 54 by a removable memory interface, such asinterface 90.

The disk drives 92, 96 transfer computer-readable instructions, datastructures, program modules, and other data for the computer 50 todifferent storage media 94, 100 for storage. A hard disk drive 76 maystore an operating system 64, application programs 66, other programmodules 70, and program data 72. These components may be the same ordifferent from operating system 80, application programs 82, otherprogram modules 84 and program data 86. The components associated withthe hard disk drive 76 may be different copies of the same elements(e.g., 64, 66, 70, 71, 71 a, 71 b, 72) that are associated with RAM 62.

The user may interact with the computer 50 through input devices such asa keyboard 106 or a pointing device 104 (i.e., a mouse). A user inputinterface 102 may be coupled to the system bus 54 to allow the inputdevices to communicate with the processing unit 51. A display devicesuch as a monitor 122 may also be connected to the system bus 54 via avideo interface 120.

The computer 50 may operate in a networked environment using logicalconnections to one or more remote computers 114. The remote computer 114may be a PC 12, a server 34, a router 20, or other common network nodeas illustrated in FIG. 1. The remote computer 114 typically includesmany or all of the previously-described elements regarding the computer50, even though only a memory storage device 116 is illustrated in FIG.2. Logical connections between the computer 50 and one or more remotecomputers 114 may include a wide area network (WAN) 112. A typical WANis the Internet. When used in a WAN, the computer 50 may include a modem110 or other means for establishing communications over the WAN. Themodem 110 may be connected to the system bus 54 via the user inputinterface 102, or other mechanism. In a networked environment, programmodules 71 depicted relative to the computer 50, may be stored in theremote memory storage device 116. By way of example, and not limitation,FIG. 2 illustrates website data and remote application programs 124 asresiding on the memory device 116. As may be appreciated, other means ofestablishing a communications link between the computer 50 and theremote computer 114 may be used.

As previously described, the method and system may build and viewdynamic documents with the use of an information management system 40 byutilizing a two-part application to create digital interactivemultimedia documents and to provide a seamless, self-guided userexperience. This is a novel medium that combines the benefits ofhyperlinked documents, like Adobe® PDF with interactive audiovisualmedia, like PowerPoint® and Flash®. This is a substantial technologicalimprovement over conventional documents that incorporate multiple mediafiles where:

-   -   a). the document is typically created in Word® and then        formatted and combined with various images before being        flattened, so that the viewer of the document is essentially        looking at a static document without links to other files;    -   b). conventional PowerPoint® presentations are employed, which        allow only limited interactivity and limited navigation through        a rigid sequence of “slide” screens with no scrolling or        zoom-in, and no ability to “hover” a new window with linked        media;    -   c). Adobe® Flash™-type technology is typically employed, which        requires a highly skilled operator to create interactive files        from whole cloth; and    -   d). conventional document and media file integration techniques        are employed and require embedding several copies of a single        media file into a final document in order to view different        portions of the embedded document or media file.

FIG. 3 illustrates a block diagram of an exemplary builder screen 150and presenter screen 152 that are utilized in building and viewinginteractive multimedia documents that incorporate references to exhibitsor other supporting documents, such as, for example, legal briefs. Thebuilder screen 150 is particularly effective in its ability to enablenon-technical people of ordinary clerical computer skills, such as, forexample, secretaries, paralegals, attorneys, managers, etc., to designand produce full-featured documents with a look and feel of thosedocuments produced by a professional graphic designer. Embodiments mayinclude a text portion 154 of a document (e.g., a legal brief, a motion,a memorandum, etc.), evidence 156 that supports a syllogistic or logicalstep of an argument presented by the text portion 154 of the document,and a variety of templates 158 that allow a user 157 at the builderscreen 150 to incorporate the evidence 156 into the text 154 to presentthe finished interactive multimedia document to a viewer 159 in apredictable and visually-flowing manner. For example, the evidence 156may be a variety of media files 160 including text 162, video 164,PowerPoint® 166, Portable Document Files 168, and other files 170 (e.g.,audio, digital images, etc.) that may be incorporated, as describedbelow, into an interactive multimedia document.

A system for building and viewing dynamic documents may include avariety of structures and components as generally described in relationto FIGS. 1-3. Therefore, the system configurations described in relationto FIGS. 1-3 may include any combination of elements described inrelation to each figure.

FIG. 4 illustrates a flowchart describing a method of one example ofbuilding a dynamic interactive multimedia document (hereinafter, “thedocument” or “the dynamic document”) 300 (FIG. 6). The method may beginwhen a user 157 creates a text portion 154 (FIG. 3) of the document 300using a word processing application, such as, for example, MicrosoftWord™ (block 200). The user 157 may utilize the builder screen 150 whichincorporates a WYSIWYG interface to provide “stay at home” navigation tocreate a distinctive view for a viewer 159 that remains located in thehome page of the document 300. This allows the user 157 to manipulatetext 154, images, and other media files 160 to achieve enhancedappearance of the document 300 through application of the designtemplates 158 and through user design choices as related to text 154,colors, and images. The text 154 remains in vector format, so thecontent remains malleable during the entire building/creation processand is easily accessible to all members of a team responsible forcreating the document 300, assuming more than one person is involved inthe creation of the document 300.

The WYSIWYG interface does not require a user 157 to write any specificcode or create hyperlinks to media files 160, but allows the user 157 tosimply drag and drop media files 160 and portions or sections of mediafiles 160 into specific locations of the document 300. The buildermodule 71 a (FIG. 2) may employ a form of markup language (e.g., SGML,XML, etc.) to describe the document 300 including a number of references302 (FIG. 6) to relevant media files 160 that include media filemetadata 172, 174, 176, 178, 180 (FIG. 3). A media file 160 may be anytype of audio, video, or text file that may be combined with the textportion 154 of the document 300, in whole or in part, as support for asyllogistic or logical step of an argument or element of an argumentthat is presented to a user who views the document 300. The media file160 may be defined or described by media file metadata 172-180 that mayidentify any aspect of the media file 160. For example, media filemetadata 172-180 may describe a specific portion of the media file 160(e.g., a paragraph within a text media file, or a segment of audio orvideo, etc.) that supports a syllogistic or logical step of an argumentbeing presented to the viewer 159 on the presenter screen 152 at aspecific location of the document 300. The media file metadata 172-180may be employed by the builder module 71 a as a “tag” or reference todescribe the specific portion of the media file 160 so that the portionmay then be recalled and formatted by the presenter module 71 b to bepresented to the viewer 159 in the presenter screen 152. Therefore,portions of the media file 160 may be described by a first type of mediafile metadata 172-180, while a format of the portion may be described bya second type of media file metadata 172-180.

To embed formatted portions of the media files 160 into the document300, with the builder module 71 a, a reference (e.g., a hyperlink) 302(FIG. 6) to a media file 160 may include both the portion of the mediafile 160 and its format (e.g., color, highlighting, underlining, audiovolume, video properties, etc). By including separate metadata tags forvarious portions and characteristics of the media file 160, thepresenter module 71 b may format the portions for viewing within thepresenter screen 152 upon the viewer 159 selecting the reference 302. Byformatting the portions of the media file 160 at the time the viewer 159selects the reference 302 while viewing the document 300, there is noneed for the document 300 to store several pre-formatted copies of eachmedia file 160. Thus, only one copy of the media file 160 may beincluded with the document 300 to present any portion of the media file160 in any format. Further, by only including one copy of the media file160, the amount of memory 62 (FIG. 2) used by the document 300 may bereduced.

In some embodiments, upon viewing the a portion of the document 300 thatincludes a reference 302 to a portion of a media file 160, the viewer159 may view the portion of the media file 160 by a passive oraffirmative action. For example, the viewer 159 may activate thereference 302 with a mouse click or roll over action, selecting therelevant media file 160 or portion of the media file from a menu, or anyother act. The reference 302 to the media file 160 may include anyamount of media file metadata 172-180 to allow the presentation of themedia file 160 or a relevant portion of the media file 304 (FIG. 7) tothe viewer 159 within the context of the argument presented by thedocument 300. Upon selection of the reference 302, the presenter module71 b may utilize the media file metadata 172-180 to format and present aportion of the media file 304 to supplement the viewer's 159understanding of a particular syllogistic or logical step of theargument that is being viewed within the presenter screen 152.

As described above, the viewer 159 is able to view a formatted, relevantportion of the media file 304 within the context of the document 300 asa whole and without pulling the viewer's focus to irrelevant portions ofthe media file 160. This “stay at home” navigation aspect is importantbecause it allows the creators 157 to prevent viewers 159 from viewingthe media files 160 randomly. Instead, viewers 159 are presented withthe formatted, relevant portion of the media file 304 content in contextand for the purpose of substantiating particular arguments discussedwithin relevant portions of the document 300. Thus, the creator 157 isable to create a tightly controlled experience for viewers 159 of thedocument 300, while still allowing the viewer 159 to interpret relevantportions of supporting media files 160. In some embodiments, the “stayat home” navigation is accomplished by causing the formatted, relevantportions of the media files 304 to be presented to the viewer 159 insuch a way that they appear to hover over the “home” portion of thedocument 300. An example of this concept is illustrated in FIGS. 6 and7, where FIG. 7 illustrates a formatted, relevant portion of a mediafile 304 (a portion of a text document 162) hovering over a main page ofa legal brief 300. The portion of the media file 304 is illustratedstylistically as a floating window over a dynamic document 300 thatappears to be slightly shaded. When the hovering formatted, relevantmedia file 304 (portion of a text document 162) is closed, the viewer159 is returned to the exact same place in the main document 300 thatthey were prior to viewing the portion 304. Thus, the viewer 159 is notrequired to use a forward and back button or to open multiple windows toview the relevant portions of the media files 304. This greatlyincreases the likelihood that the viewer 159 will not become lost ordistracted while viewing the supporting media files 160. Instead, theviewer 159 will activate a reference 302 to refer to the relevantportions of the media files 304 only in conjunction with specificportions of the dynamic document 300. Thus, the relationships betweenthe portions of the media files 304 and the points or argumentsdiscussed in the dynamic document 300 remain closely connected.

The creator 157 may also prevent the viewer 159 from altering thecontent of the media files 160. In other words, the content is locked bythe creator 157, but in a substantially different way from a PDFdocument.

Referring again to FIG. 4, the creator is provided the ability to applydesign template(s) or graphical template(s) 158 (FIG. 3) to the text 154to control the look and style of one or more of the text 154 andrelevant portions of the media files 304 (block 202). The templates 158may include, for example, borders, colors, positions of media window(s),shading, highlighting color, etc. To control the formatting of theportions of the media files 304, the templates 158 may employ metadata172-180, as previously described. It is envisioned that a defaulttemplate 158 could be implemented as well as a number of alternativetemplates that a creator 157 could choose from when building a dynamicdocument 300.

The creator 157 may also be permitted to generate concept headings 306(block 204), such as those shown in FIG. 6. The concept headings arepurely optional and may be generated at any time during the creation ofthe dynamic document 300. These concept headings 306 may be linked toand correspond to particular sections within the dynamic document 300 sothat, upon selection by the viewer 159, the presenter module 71 bpresents a portion of the document 300 that corresponds to the heading306.

The creator 157 is also permitted to identify the media files 160 toinclude in the dynamic document 300 and/or make available to a viewer159 of the dynamic document 300 (block 206) using the builder module 71a (FIG. 2). The creator 157 is provided the ability to browse and selectmedia files 160 and specific portions of the media files 304 that arethen linked to specific portions of the dynamic document 300. Forexample, the creator 157 is able to locate text from documents 162,segments of video 164, slides from PowerPoint presentations 166,portions of PDF documents 168, and other relevant files 170 in contextand in proximity to the facts and arguments discussed in the text of thedynamic document 300. Therefore, the creator 157 can strategically placereferences 302 to relevant portions of the media files 304 in thedynamic document 300 to create the most persuasive viewer experience. Inother words, after one or more media files 160 have been identified, thecreator 157 may determine the relationships between specific portions ofthe media files 304 (text, video, images, graphics, voice-narratedgraphics, animation, etc.) and text portions 154 of the dynamic document(block 210).

The method may then allow the creator 157 to determine how themultimedia file will be displayed based on its relationship to the text154 being viewed (block 212). This will include both the specificportion of the multimedia file 304 as well as the visual appearance ofthe portion. The dynamic document 300 may also include mediaorganization information that may contain both media categories, suchas, for example, exhibits, cases, statutes, demonstratives, etc., andthe location of the media (pointers to the specific folders where thecontent is stored). This media information, if included, may be used togenerate a table of contents view and to search media files 160 orrelevant portions of the media files 304 by content type. The dynamicdocument 300 may also include graphical template information anddocument preferences that may contain viewing preferences for the textportion 154 of the document 300 including, for example, text font sizeand style, line spacing, color scheme, size, color, and appearance ofvarious graphical elements, appearance of table of contents, etc. Thedynamic document 300 may then be merged using XML style logic that mayinclude the document text 154, links to media files 160, media filemetadata 172-180 describing a location and format of the relevantportion of the media file 304, graphical template information, documentpreferences, and media organization information (block 214).

FIG. 5 illustrates a flowchart describing an alternative method ofbuilding a dynamic interactive multimedia document 300 (FIG. 6). Themethod may begin when a user 157 (FIG. 3) creates a text portion 154 ofa document using a word processing application, such as, for example,Microsoft Word™ (block 250). The user 157 may utilize the builder screen150 of the builder module 71 a (FIG. 2) that incorporates a WYSIWYGinterface to provide “stay at home” navigation, as described above, tocreate a distinctive viewer 159 experience that remains located in thehome page of the document 300. The creator 157 of the dynamic document300 is provided the ability to apply design template(s) or graphicaltemplate(s) 158 to one or more of the text 154 and the portions of themedia files 304 to control their look and style (block 252). This mayinclude, for example, borders, colors, positions of media window(s),shading, highlighting color, etc. The creator 157 may also be permittedto generate concept headings (block 254), such as those shown in FIG. 6.The concept headings are purely optional and may be generated at anytime during the creation of the dynamic document. These concept headingsmay be linked to and correspond to particular sections within thedynamic document so that a viewer 159 is taken directly to theappropriate portion of the document 300 after selecting the appropriateheading.

Using the builder module 71 a (FIG. 2), the creator 157 is alsopermitted to identify the media files 154 to include in the dynamicdocument 300 and/or make available to a viewer 159 of the dynamicdocument 300 (block 256). The creator 157 is provided the ability tobrowse and select a first media file 160 (e.g., a text document 162) anddetermine the relevant portion 304 and format of the media file. Thebuilder module 71 a then associates metadata 172-180 describing themedia file relevant portion 304 and its format with the media file 160(e.g., 172). The creator 157 may then use the builder module 71 a tocreate a reference 302 that includes the metadata 172 and to associatethe reference 302 with a portion of the dynamic document 300 that wouldbe supported by the selected portion 304. In other words, the creator157 is able to locate images and links to digital documentary or videoevidence, etc. that support a particular syllogistic or logical step ofthe argument presented by the text 154 of the document 300 to create areference 302 to the relevant portion of the media file 304. The creator157 then places the reference 302 in proximity to the facts andarguments discussed in the text 154 of the dynamic document 300.

Upon selection of the reference 302 by the viewer 159, the presentermodule 71 b interprets the metadata 172 of the reference 302 to presentthe formatted, relevant portion of the media file 304. Therefore, thecreator 157 can strategically place images and linked evidence in thedynamic document 300 to create a persuasive viewer experience. Morespecifically, the creator 157 is provided with the ability to access thefirst multimedia file (e.g., 162), while still in its native format,that corresponds with a concept described in the first portion of thetext 308 (FIG. 6) (block 260). This is illustrated in FIG. 6 which showsa “home” page of a dynamic legal brief document 300 that includes areference 302 or hyperlink to “Plaintiff Exhibit P15.” This reference302 is linked to the first multimedia file 162 which is an issued U.S.patent, as shown in FIG. 7.

After the creator 157 identifies the first multimedia file 162 (the U.S.patent in FIG. 7), the creator 157 may determine a first portion of thefirst multimedia file 304 that provides evidence or otherwise supportsthe concept or logic step described in the first portion of text 154from the “home” page of the document 300. The creator 157 may alsodetermine the appearance or format of the first portion of the firstmultimedia file 304 (block 262). For example, the creator 157 may chooseto present an enlarged, “zoomed in” view 310 (FIG. 7) of the relevantportion 304, a highlighted view 315 of the relevant portion 304, acombination of zooming and highlighting 320 (FIG. 8), or otherformatting. In some embodiments, the viewer 159 may also be able to viewother portions of the media file 160 by, for example, scrolling to otherpages of the file 160 (as shown in FIG. 9) or by selecting a “previous”325 or “next” 330 button (FIGS. 7-12). The viewer 159 may alsomanipulate the displayed media file 304 by zooming in 332 and zoomingout 334. The viewer 159 may close the displayed media file 304 byactivating the “close” button 336. This concept is illustrated in FIGS.7-9 which show the same first portion of a multimedia file presented toa viewer in different styles.

The creator 157 of the dynamic document may use the builder module 71 a(FIG. 2) to generate a second link 344 including metadata 172 describinga formatted, second portion of the first media file 304. The second linkmay allow the viewer 159 to access to the first multimedia file 162 thatcorresponds to a concept described in a second portion of the text inthe dynamic document 340 (FIG. 6) (block 264). The creator 157 maydetermine a second portion of the first multimedia file 162 to displaythat corresponds with the concept described in the second portion oftext from the “home” page of the text 340, as well as the appearance ofthe second portion of the first multimedia file (block 266). FIG. 10illustrates a second portion of the first multimedia file (the U.S.patent) 344. It is also noted that both the first and second portions ofthe multimedia file (304, 344) hovering over the “home” page of thedynamic document 300 include navigation buttons (325, 330, 332, 334,336) that allow the viewer to progress to other portions of themultimedia file 162, while maintaining the relationship to the “home”page of the dynamic document 300 (which changes as the viewer progress)without relocating the viewer to the linked file 162.

The builder module 71 a may also allow the creator 157 to access asecond multimedia file, while still in its native format, thatcorresponds with a concept described in a third portion of the text 350(FIG. 11) (block 270). This is illustrated in FIG. 12 which shows a“home” page of a dynamic legal brief document 300 that includes a linkto “Deposition of Dr. Frank Hanson as page 21.” This icon is linked to asecond multimedia file which is a video 164, as shown in FIG. 12. Thevideo 164 remains in its native format and is interpreted by a videomodule of the presenter module 71 b rather than being flattened into aPDF document.

After the creator 157 identifies the second multimedia file (the video164 in FIG. 12), the creator may determine a first portion of the secondmedia file 358 to display that corresponds with the concept described inthe third portion of text from the “home” page of the text 350, as wellas the appearance of the portion of the second multimedia file (block272). The creator 157 may then employ the builder module 71 a togenerate a hyperlink 354 including metadata 174 describing the firstportion of the second media file 358 (including format, location withinthe second media file 164, start and stop times, etc.). Upon selectionof the hyperlink 354 by the viewer 159, the presenter module 71 b mayutilize the metadata 174 to present the formatted first portion of thesecond media file 358, as described above in relation to FIGS. 4 and 5.

The builder module 71 a may also allow the creator 157 to access a thirdmultimedia file, while still in its native format, that corresponds witha concept described in a fourth portion of the text 360 (FIG. 11). Thisis illustrated in FIG. 12 which shows a “home” page of a dynamic legalbrief document 300 that includes an image 362 entitled “What is aPhospholipid?” This image 362 is linked to a third multimedia file whichis a voice-narrated graphics file 170, as shown in FIG. 13. Thevoice-narrated graphics file 170 remains in its native format and is notflattened into a PDF document.

After the creator 157 identifies the third multimedia file (thevoice-narrated graphics file 170 in FIG. 12), the creator may determinea first portion of the third media file 358 to display that correspondswith the concept described in the fourth portion of text from the “home”page of the text 350, as well as the appearance of the portion of thethird multimedia file (block 272). The creator 157 may then employ thebuilder module 71 a to generate a hyperlink 366 including metadata 180describing the first portion of the third media file 368 (includingformat, location within the third media file 170, etc., as previouslydescribed). Upon selection of the hyperlink 366 by the viewer 159, thepresenter module 71 b may utilize the metadata 180 to present theformatted first portion of the third media file 368 (FIG. 13), asdescribed above in relation to FIGS. 4 and 5.

When the media file 160 includes playback features (e.g., audio, video,voice-narrated graphics, etc.) the presenter screen 152 may incorporatecontrols 370 for the viewer 159 to manipulate the playback of the mediafile 160. In some embodiments, the controls 370 may include a volumecontrol, a play control, and a pause control.

FIG. 14 is an exemplary illustration of a dynamic document 300 in whicha viewer has conducted a key word search (e.g., the word “liposomes”).Because the text of the dynamic document remains in its native format(e.g., vectored or XML format), the identified key words 364 are easilyidentified and displayed with highlighting. This is substantially moreaccurate and powerful than a PDF document that is linked to an OpticalCharacter Recognition (OCR) file.

The presenter module 71 b may also include a note taking module thatallows a viewer 159 to enter text and audio notes to refer and explainparticular points within the document 300 upon later viewing.

Although the forgoing text sets forth a detailed description of numerousdifferent embodiments, it should be understood that the scope of thepatent is defined by the words of the claims set forth at the end ofthis patent. The detailed description is to be construed as exemplaryonly and does not describe every possible embodiment because describingevery possible embodiment would be impractical, if not impossible.Numerous alternative embodiments could be implemented, using eithercurrent technology or technology developed after the filing date of thispatent, which would still fall within the scope of the claims.Accordingly, it should be understood that the methods and apparatusdescribed herein are illustrative only and are not limiting upon thescope of the claims.

Thus, many modifications and variations may be made in the techniquesand structures described and illustrated herein without departing fromthe spirit and scope of the present claims.

This invention can be applied (but not limited) to: legal briefs, expertreports and declarations, including electronic and hosted submissions tocourts; sales presentation and proposal bid packages; buildingmultimedia web sites with linked media; reporting scientific ortechnical findings; publishing technical or operation manuals; or eventhe storage and retrieval of medical records including test images andvideo files.

What is claimed is:
 1. A computer-implemented method for building andviewing an interactive multimedia document comprising: determiningmetadata describing a location for a media file portion within a mediafile; embedding a reference within a text portion of the interactivemultimedia document, the reference including the metadata; interpretingthe metadata upon selection of the reference; and displaying the mediafile portion over the interactive multimedia document based on theinterpreted metadata.
 2. The computer-implemented method of claim 1,wherein the media file includes one or more of a text file, an audiofile, or a video file.
 3. The computer-implemented method of claim 1,wherein the metadata also specifies how the media file portion isdisplayed over the interactive multimedia document.
 4. Thecomputer-implemented method of claim 1, wherein the media file includesa file type including a PowerPoint® presentation, a portable document, aWord® document, and a Windows® Media file.
 5. The computer-implementedmethod of claim 4, further comprising visually formatting the media fileportion and displaying the visually formatted media file portion uponselection of the reference.
 6. The computer-implemented method of claim1, wherein the interactive multimedia document is one or more of a legalbrief or a memorandum.
 7. The computer-implemented method of claim 1,wherein selection of the reference causes a processor to interpret themetadata.
 8. The computer-implemented method of claim 1, whereinselection of the reference includes one or more of a roll over action ora clicking action of a mouse device.
 9. The computer-implemented methodof claim 1, wherein the text portion is in a vector format.
 10. Thecomputer-implemented method of claim 1, wherein the metadata describes avisual format of the media file portion.
 11. The computer-implementedmethod of claim 10, wherein the visual format includes one or more of aborder, a color, a position of a media window displaying the media fileportion, shading, or a highlighting color.
 12. A computer-readablestorage medium including computer-executable instructions stored thereonthat, when executed, cause a computer to: determine metadata describinga location for a media file portion within a media file; embed areference within a text portion of the interactive multimedia document,the reference including the metadata; interpret the metadata uponselection of the reference; and display the media file portion over theinteractive multimedia document based on the interpreted metadata. 13.The computer-readable storage medium of claim 12, wherein the metadataalso describes a visual format of the media file portion, the visualformat including one or more of a border, a color, a position of a mediawindow displaying the media file portion, shading, or a highlightingcolor.
 14. The computer-readable storage medium of claim 12, whereinselection of the reference includes one or more of a roll over action ora clicking action of a mouse device and selection causes a processor tointerpret the metadata.
 15. The computer-readable storage medium ofclaim 12, further comprising instructions that, when executed, cause thecomputer to: determine further metadata describing another location foranother media file portion within the media file; and embed anotherreference within another text portion of the interactive multimediadocument, the reference including the other metadata; wherein theinteractive multimedia document includes a single copy of the mediafile.
 16. A system for building and viewing an interactive multimediadocument at a computer, the system comprising: a text document datafile; a multimedia data file; one or more computers including: one ormore processors configured to execute computer readable instructions;and a memory having stored therein processor-executable computerreadable instructions, the instructions comprising: a builder module todetermine metadata describing a location for a media portion within themultimedia data file and to embed a reference within the text documentdata file, the reference including the metadata; and a presenter moduleto display a text document corresponding to the text document data file,to interpret the metadata upon selection of the embedded referencewithin the displayed text document, and to display the media portionover the displayed text document based on the interpreted metadata. 17.The system of claim 16, wherein the metadata also describes a visualformat of the media portion, the visual format including one or more ofa border, a color, a position of a media window displaying the mediaportion, shading, and a highlighting color.